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ABSTRACT 

With the currently growing interest in social network services, many college courses use social network services 
as platforms for discussions, and a number of studies have been conducted on the use of social network analysis 
to measure students’ participation in online discussions. This study aims to demonstrate the difference between 
counting posts and social network analysis of posts as a form of learners’ participation in online discussions. To 
accomplish the goal, the study analyzed students’ participation in Facebook discussions using the two methods 
and compared their results with those of MANOVAs. The between-group difference was significant when 
participation was measured by closeness centrality, but it was not significant when participation was measured 
by the number of posts. Although whether participation was measured by closeness centrality or by the number 
of posts did not make a significant difference in terms of learners’ self-regulated learning level, the observed 
power of the closeness centrality measurement was higher than that of the number of posts measurement. These 
findings imply that it is important for a relational analysis to consider participation in terms of, not only the 
interaction between actors, but also closeness centrality, by comparing the two measurement methods. 
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INTRODUCTION 

There is a growing interest in social network services (SNSs), and accordingly, many college courses use SNSs 
as platforms for discussion. A number of studies have used social network analysis (SNA) to measure students’ 
participation in online discussions (De Laat, Lally, Lipponen, & Simons, 2007; Lipponen, Rahikainen, Lallimo, 
& Hakkarainen, 2001; Lipponen, Rahikajnen, Hakkarainen, & Palonen, 2003; Stakias, Psoras, & Glykas, 2013; 
Tomsic, & Suthers, 2006). SNA aims to describe structural patterns of relationships among social actors, groups, 
and organizations and their implications (Hatala, 2006; Mitchell, 1969; Reffay & Chanier, 2003; Scott, 2013; 
Wasserman & Faust, 1995). It is an effective means for collecting, storing, and managing big data and 
analyzing and facilitating data visualization, which explains the relationship of exponential information 
(Stemitzke, Bartkowski, & Schramm, 2008; Suh & Shin, 2012). A number of previous studies have considered 
SNA to be a theory or analytic technique, or an interdisciplinary methodology for identifying social structural 
variables and properties. And Hoff, Raftery and Handcock (2002) said that SNA can demonstrate the 
relationships among interactive actors in an unobserved social space and provide more information such as 
actor’s power, prestige and information authority in networks, which makes it a sophisticated statistical and 
useful research tool (Aviv, Erlich, Ravid, & Geva, 2003; Garton, Haythomwaite, & Wellman, 1997; Martinez, 
Dimitriadis, Rubia, Gomez, Garrachon, & Marcos, 2002; Wasserman & Faust, 1995). 

As the importance about online discussion environment as new platforms for discussion has become more and 
more, there has been much empirical evidence of the usefulness of SNA methods for analyzing online networks, 
specifically, e-leaming social awareness as well as evaluation technique in collaborative e-leaming 
(Lambropoulos, Faulkner, & Culwin, 2012), the response relations among participants in asynchronous online 
discussions (Aviv et al, , 2003), and the participation in online-learning (Kim & Park, 2009; Park & Choi, 2011; 
Suh & Shin, 2012). Specifically, learner’s participation- one of the factors predicting learners’ achievements-, 
expressed as posts and/or comments in online discussion environment was influenced by learner’s interactions 
(Moore & Marra, 2005; Jung, Choi, Lim, & Leem, 2002). So SNA method which is tools considering actors’ 
interactions (including direct and indirect connections) can make it easy to comprehend the influence of each 
actor in networks. In this regard, it is highly possible that using SNA indexes can lead to conclude unprecedented 
implications which could not be revealed by using a simple count of meaningful posts because of SNA indexes 
demonstrating interactive relations between actors. Consequently, this study aims to demonstrate the difference 
between counting meaningful posts and SNA of posts as a form of learners’ participation in online discussions 
by using the closeness centrality of posts to analyze authentic relations, interactions, and the position of actors. 


Copyright © The Turkish Online Journal of Educational Technology 


11 





TOJET: The Turkish Online Journal of Educational Technology - January 2016, volume 15 issue 1 



Most research reported that SNA indexes(especially closeness centrality) are able to analyze and enhance the 
deeper understanding of research based on interaction, however, how deep and wide in the research is not 
definite. Therefore, research to identify the difference between traditional research method and SNA method are 
required, and the research question is, “Are counting of meaningful posts and SNA of posts as a form of 
learners’ participation different in the analysis of online discussions?” 

LITERATURE REVIEW 
Social Network Analysis 

SNA, a strategy for investigating social networks or relations, can be applied in many fields as a new approach 
(Borgatti, Mehra, Brass, & Labianca, 2009; Otte & Rousseau, 2002; Stakias et al., 2013; Wasserman & 
Galaskiewicz, 1994). Some researchers have examined SNA from various aspects, including theories, methods, 
software, and the research paradigm, from an alternative metaphor to an analytic approach (Scott & Carrington, 
2011; Stakias et al, 2013), while others have perceived it as a methodology. Whereas research in past decades 
focused on theoretical assumptions, recent studies have regarded SNA as a research methodology for 
systematically analyzing complex social-structure properties originating from the interdependence underlying 
function of social relations among analytic objects (Galaskiewicz & Wasserman, 1993; Makagon, McCowan, & 
Mench, 2012; Pema, Marra, & Napolitano, 2008; Shim & Lee, 2008; Wasserman & Faust, 1995). By focusing 
on relationships among actors rather than on each actor, SNA can analyze an organizational topological structure 
and its diffusion progress (Scott, 2013; Wasserman & Faust, 1995). In other words, it can provide an 
understanding of the network’s features through link patterns, the number of links, and the structural 
concentration, not with attribute variables, but with relational variables. Also, visualization with an informal or 
formal network structure is useful for people who conduct research on big data (Pema et al., 2008). Considering 
the environments, it is necessary to deal with both the numeric comparison and the visualization of data. 

The elements of analysis include connection, centrality, cohesion, and equivalence. Researchers mainly use 
density of connection and centrality as analytic techniques (Abbasi, Hossain, Uddin, & Rasmussen, 2011; 
Enriquez, 2008; Hamulic & Bijedic, 2009; Hawe & Ghali, 2008; Rice, Tulbert, Cederbaum, Adhikari, & 
Milbum, 2012). A node’s connection is the number of neighboring nodes; it is the primary index that describes 
the node’s characteristic in the network. It is possible to analyze a connection using concepts such as degree, 
density, reciprocity, and shortest path, to analyze the fundamental relationships with nodes and links. SNA 
focuses on relational attributes and considers in-degree and out-degree. The number of in-degrees is the number 
of lines directed toward the node, and the number of out-degrees is the number of lines directed to other points. 
According to graph theory, an in-degree is placed in the column of a matrix, whereas an out-degree is placed in 
the row. Centrality is used to measure the basis of a degree (Scott, 2013), the stmcture of which depends on 
certain criteria not based on the median or a node attached to many links. The aim of centrality stmcture analysis 
is to identify one of the most important nodes in a network and to investigate critical nodes for determining the 
degree of centralization. The components of centrality include degree centrality, closeness centrality, and 
betweenness centrality. Generally, the shorter the distance to other nodes is, the higher the closeness centrality is. 
In addition, closeness centrality indicates multiplicative inverse proportionality to distance to other nodes. The 
concept of a high level of closeness centrality is applied to all other nearby actors, indicating easy and rapid 
accessibility to other actors with minimum efforts (Wasserman & Faust, 1995). 

Participation in online discussion environment 

Online environment as new platforms for discussion makes learners participate in discussion activity more and 
has a positive effect on high academic achievements, retention and transfer of learning. Besides, online 
discussion is beneficial for learners in that the learning environment is not influenced by time and location unlike 
traditional learning method. Therefore, previous studies (Berge, 1996; Dennen, 2001; Johnson & Johnson, 2000; 
Yellen, Winniford, & Sanford, 1995) suggested stmcture of groups, characteristics of learners, the role of tutor 
(or instmctor) and strategies promoting discussion learning affect learner’s participation in online discussion 
environment since that participation is closely associated with academic achievements. These research have been 
studied by quantitative and/or qualitative method, and also SNA indexes (Aviv et al, 2003; Lambropoulos et al, 
2012; Moore & Marra, 2005; Jung et al, 2002). 

Above all, since closeness centrality is measured by direct and indirect links among nodes found in an efficient 
organization (Shim & Lee, 2008), there have been a number of previous studies on the closeness centrality of 
participation in the online environment. Suh and Shin (2012) used closeness centrality providing a standardized 
value and a quantitative indication of concentration in online discussion activity, whose participation is an index 
that enables a multidimensional understanding of participation in learning as well as an analysis of interactions 
that take place among learners. Kim and Park (2009) used centrality of interaction on a Web bulletin board in 
order to analyze interdependency among learners in the Web environment. Interaction on a Web bulletin board is 
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represented by the instructor’s feedback, and its centrality is an index that affects learning, which facilitates the 
assessment of interactions as well as with whom learners are exchanging opinions and whose opinions are 
influential. Park and Choi (2011) viewed centrality in the number of posts as representing the level of 
participation and argued that analyzing the relational attributes of the discussion environment, such as the 
relationships among central and surrounding individuals, can be helpful toward understanding learners’ 
characteristics. Therefore, it is necessary to combine conventional statistical research methodology with SNA or 
content analysis in order to identify new findings with profound implications and to broaden the context of 
analysis (Park & Choi, 2011) and allows a better understanding of the evolution process in online communities 
(Bae, Seo, & Baek, 2010). Among social network indices, the centrality of participation also has a significant 
effect on quantitative achievements (Cheong & Corbitt, 2009; Cho, Gay, Davidson, & Ingraffea, 2007; Russo & 
Koesten, 2005), providing a perspective of actual relationships and interactions rather than quantified numerical 
data. 

METHOD 

Participants and Treatment 

To illustrate the approach, this paper analyzes the data drawn from a previous study that examined the effects of 
learners’ self-regulated learning (SRL) skills and the instructor’s feedback on learners’ achievement and 
participation in Facebook discussions (Park & Lee, 2013). The experiment was implemented in two classes, 
which were taught by the same instructor with the same contents. One class was a control group not provided 
with instructor feedback, and the other class was an experimental group that was provided with instructor 
feedback. Then, based on the measured levels of all the participants’ SRL, each class was divided into two 
groups representing high and low SRL, respectively. Thus, the participants were assigned to four groups: (1) 
feedback-high SRL, (2) feedback-low SRL, (3) non feedback-high SRL, and (4) non feedback-low SRL. The 
study analyzed the data of 108 participants who completed a self-regulation survey and posted meaningful posts. 

Measurement Instruments and Procedure 

In the study, the learners’ levels of SRL skills and achievement were drawn from Park and Lee’s (2013) data. 
However, their participation level was newly measured. The number of meaningful posts was determined by the 
meaning unit technique, which was widely used in the message analysis. “Meaning unit,” as the unit of analysis, 
is a unit of idea extracted from contents and contains a single item (Budd & Donohue, 1967). After dividing the 
data into units of analysis, the researchers measured the number of posts, excluding the number of extraneous 
posts related to contents. Together, they analyzed the meaning units to achieve reliability. When there were 
differences in opinion, the researchers discussed the analysis and made adjustments where necessary. 

The researchers analyzed the participants’ meaningful posts regardless of their length. Also, out-closeness 
centrality was calculated by the length of shortest paths that a specific actor posted meaningful posts to other 
actors. To measure out-closeness centrality, the meaningful posts’ matrix of between participants was figured out 
at out-closeness centrality vector using NetMiner 3.0. 


Pre Test 

- SeSf-reguSated learning: 
cognition St meta- 
cognition inspection 

* Self-regulated learning 
leveI(high/low) division 

- Intervention group 
division(feedback/nonf 
eedback) 


Learning Activity 
& Intervention 

•Join in Facebook 

* Discussion in Facebook 

* Provide feedback on the 
experimental groups 
according to the time of 
discussion 


Post Test 

- Measuring participation 

M a • # of meaningful posts 

M s : Outcloseness centrality of M A 
* Report(task) evaluation 



Figure 1: Revised from Park and Lee (2013). The Effects of Learners’ Self-regulated Learning 
Skills and Instructor’s Feedback on the Learners’ Achievement and Participation in 
Facebook®-based Discussion of a Higher Education Setting. The Journal of 
Educational Information and Media, 91(2), 229-251. 


Analysis 

To investigate the difference between SNA of posts and counting of meaningful posts, data were analyzed using 
NetMiner 3.0 and SPSS Statistics 18. For each measurement method that used feedback and level of 
self-regulation as independent variables with participation and achievement as dependent variables, descriptive 
statistics and multivariate analysis of variance were performed. The results of MANOVA depending on the 
measurement method were compared. For all statistical analyses, a level of significance of .05 was chosen. 
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RESULTS 

Result of counting and meaningful posts 

Table 1 shows the descriptive statistics of the data regarding instructor feedback, learners’ achievement 
according to level of SRL skills, logarithmic value of participation level, and number of cases. Measurement A is 
a semantic analysis of Park and Lee’s data pertaining only to the meaningful comments on each post. The 
maximum level of achievement was set at 20, and the average level was 16.67. 


Table 1: Descriptive Statistics of Measurement A 






Measurement A 




Feedback 

SRL 


(counting of meaningful posts) 

Achievement 

n 


Raw 

Log 







M 

SD 

M 

SD 

M 

SD 


High 

36 

5.06 

3.71 

0.60 

0.30 

17.24 

1.70 

Feedback 

Low 

25 

3.12 

2.49 

0.40 

0.29 

16.64 

2.03 


Sum 

61 

4.26 

3.38 

0.52 

0.31 

16.99 

1.85 


High 

21 

6.38 

5.29 

0.66 

0.38 

16.96 

1.96 

No Feedback 

Low 

26 

4.08 

2.26 

0.55 

0.25 

15.65 

1.56 


Sum 

47 

5.11 

4.03 

0.60 

0.31 

16.24 

1.85 

Sum 


108 

4.63 

3.69 

055 

0.31 

16.67 

1.88 


MANOVA was conducted to determine if the instructor feedback and level of SRL skills had an effect on 
learners’ achievement and participation. In Measurement A, the result of Levene’s homogeneity of variance test 
indicated that the difference in covariance between two dependent variables-achievement (F = 1.201, p = .313) 
and the logarithmic value of participation (F = 1.992, p = .120)-was not statistically significant at the 
significance level of .05, satisfying MANOVA assumptions. Furthermore, Box’s M test on the covariance 
matrices of the dependent variables yielded Box’s M of 7.282 (F = .780, p = .635), passing the homogeneity test. 
The results of MANOVA are shown in Table 2. 


Table 2: MANOVA results of Measurement A 




Wilks’s k 

F 

P 

NCP 

Observed 

Power 

Measurement 

A 

Feedback 

.922 

4.330* 

.016 

8.661 

.740 

A 

(counting of 

SRL 

.902 

5.626* 

.005 

11.251 

.850 

meaningful 

posts) 

Feedback 

*SRL 

.979 

1.124 

.329 

2.247 

.243 


*p < .05 


Result of the closeness centrality of posts 

Table 3 shows the descriptive statistics of the data regarding instructor feedback, learners’ achievement 
according to the level of SRL skills, average and standard deviation of the closeness centrality vector value, and 
number of cases. Measurement B is the closeness centrality of Measurement A’s out-degree. Measurement B 
includes the interaction between actors, which is different from Measurement A, as well as the concept of 
closeness centrality based on the geodesic distances among actors. 

Among the closeness centralities associated with participation, the out-closeness centrality vector value was used 
for Measurement B. In a network with directionality, closeness centrality that represents status and influence 
among actors in networks can be categorized into in-closeness centrality and out-closeness centrality. 
In-closeness centrality signifies the shortest distance from other actors to a specific actor, whereas out-closeness 
centrality is the shortest distance from a specific actor to others. In other words, closeness centrality is the 
shortest distance between actors, and in-closeness and out-closeness are categorized based on their direction. The 
data used in this study involved the number of posts left by each participant for other participants in an online 
environment, which is associated with out-closeness centrality directed from a specific actor to others. This is 
because learners’ participation level in the online discussion environment was influenced by the instructor’s 
feedback and other learners’ direct and indirect effects; therefore, this study used out-closeness centrality, which 
provided a standardized value to analyze multidimensionally in detail in the online discussion activity. As a 
result, the out-closeness centrality vector value of the number of posts written by participants was used to 
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indicate the level of participation in this study. The maximum level of achievement was identical with the scores 
used in the Measurement A analysis. 

_Table 3: Descriptive statistics of Measurement B_ 

Measurement B 


Feedback 

SRL 

n 

In-degree 

Out-degree 

Closeness 

centrality 

/irmcvcmcill 

M SD 

M 

SD 

M 

SD 

M 

SD 


High 

36 

4.14 

2.99 

4.06 

3.71 

0.34 

0.15 

17.24 

1.70 

Feedback 

Low 

25 

2.24 

1.79 

2.12 

2.49 

0.15 

0.13 

16.64 

2.03 


Sum 

61 

3.36 

2.71 

3.26 

3.38 

0.27 

0.17 

16.99 

1.85 


High 

21 

5.43 

3.88 

5.38 

5.29 

0.41 

0.20 

16.96 

1.96 

No 

Feedback 

Low 

26 

3.23 

2.29 

3.08 

2.26 

0.28 

0.15 

15.65 

1.56 


Sum 

47 

4.21 

3.26 

4.11 

4.03 

0.34 

0.18 

16.24 

1.85 

Sum 

108 

3.73 

2.98 

3.63 

3.69 

0.30 

0.18 

16.67 

1.88 


MANOVA was also conducted to determine if the instructor feedback and level of SRL skills had an effect on 
learners’ achievement and participation. In Measurement B, the result of Levene’s homogeneity of variance test 
indicated that the difference in covariance between two dependent variables—achievement (F = 1.201, p = .313) 
and the closeness centrality of participation (F = .738, p = .532)—was not statistically significant at the 
significance level of .05, satisfying the assumptions. Box’s M test on the covariance matrices of the dependent 
variables yielded Box’s M of 12.434 (F = 1.331, p = .214), passing the homogeneity test. The results of 
MANOVA are shown in Table 4. 


Table 4: MANOVA results of Measurement B 




Wilks’s l 

F 

P 

NCP 

Observed 

Power 

Measurement 

Feedback 

.873 

7.501* 

.001 

15.002 

.938 

B 

(closeness 

centrality) 

SRL 

.776 

14.895* 

.000 

29.790 

.999 

Feedback 

*SRL 

.975 

1.346 

.265 

2.692 

.285 


*p < .05 


Comparison of closeness centrality with counting of meaningful posts 

Table 5 shows the comparison of two MANOVAs. There were statistical differences between Measurements A 
and B. First, MANOVA confirms that there was no significant difference in participation measured by the 
counting of meaningful posts (Measurement A) based on the provision of instructor feedback (F = 3.086, p 
= .082), but participation measured by closeness centrality (Measurement B) based on the provision of instructor 
feedback differed significantly (F = 10.014, p = .002). Second, Measurements A and B were significantly 
different in terms of participation based on learners’ SRL level. However, the observed powers of Measurements 
A and B were .763 and .999, respectively. 

Table 5: Comparison of univariate tests of closeness centrality with those of number of posts (n = 108) 




Type III 
SS 

df 

MS 

F 

P 

Partial 

o 2 

Observed 

Power 

Measurement Feedback 

A 

Achievement 

10.388 

1 

10.388 

3.196 

.077 

.030 

.425 

Participation 

.280 

1 

.280 

3.086 

.082 

.029 

.413 

/ \ 

(counting of SRL 

fnponiii o"m i 1 

Achievement 

23.607 

1 

23.607 

7.263* 

.008 

.065 

.761 

Participation 

.663 

1 

.663 

7.297* 

.008 

.066 

.763 

meanmgiui 

posts) Feedback 

F ' *SRL 

Achievement 

3.349 

1 

3.349 

1.030 

.312 

.010 

.171 

Participation 

.056 

1 

.056 

.611 

.436 

.006 

.121 

Measurement Feedback 

Achievement 

10.388 

1 

10.388 

3.196 

.077 

.030 

.425 
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B 

(closeness 


Closeness 

centrality 

.244 

1 

.244 

10.014* 

.002 

.088 

centrality) 


Achievement 

23.607 

1 

23.607 

7.263* 

.008 

.065 


SRL 

Closeness 

centrality 

.637 

1 

.637 

26.153* 

.000 

.201 


Feedback 

*SRL 

Achievement 

Closeness 

centrality 

3.349 

.032 

1 

1 

3.349 

.032 

1.030 

1.300 

.312 

.257 

.010 

.012 


*p < .05 


.880 

7m 

.999 

7m 

.204 


In conclusion, it is important for relational analysis to consider participation in terms of, not only interaction 
between actors, but also closeness centrality, by comparing the two measurement methods. Because the 
closeness centrality of SNA focuses on relationships between actors instead of each individual actor and reflects 
social phenomena for analyzing the implications of relation, SNA should be widely applied in 
intra-organizational actors’ positions as well as in inter-organizational network. 

DISCUSSION 

The major goal of quantitative research focused on variables is to test a theory and offer a broad explanation 
about the prediction. However, a variety of factor-control variables, measuring variables, intervention variables, 
and moderating variables that affect the experimental environment are attribute variables, which only consider 
the direct connection between variables. Research focusing on attribute variables can provide a solution to a 
problem, but it is not an effective method that takes relational attributes into account. In accordance with the 
state of complex society and environment, it is necessary to comprehend unseen things and conduct sophisticated 
analysis. Therefore, this study examined the difference between SNA of posts and counting meaningful posts as 
forms of learners’ participation in online discussion. 

The results of this study can be summarized in three parts. 

First, the participation measured by closeness centrality based on the provision of instructor’s feedback was 
significant, but measured counting of meaningful posts was not significant. Since the closeness centrality is 
associated with relationship between an individual and other members of the network directly and unmediatedly 
(Cho et al, 2007), in this regard, the result indicated that instructor’s feedback have a positive effect on other 
participants who is ‘indirectly’ connected with a participant in whole network space That is, SNA indexes are 
able to avoid failing to consider influence of instructor’s feedback in online environment unlike numerical value 
by counting of meaningful posts, the conclusion is that instructor’s feedback in online environment widely 
promotes learner’s participation as a whole. 

Second, both participation measured by closeness centrality of posts and participation by counting meaningful 
posts were significantly different based on the level of learners’ SRL; and participation measured by closeness 
centrality of posts was more significantly than that by counting posts simply. In addition, a number of previous 
studies have used the closeness centrality of participation to gain a better understanding of numerous social 
networks in the online environment (Bae et al., 2010; Suh & Shin, 2012; Park & Choi, 2011; Rice Doran, Doran, 
& Mazur, 2011). The level of learners’ participation can vary according to diverse factors, such as gender, 
background, cultural traits, prior training and education, and prior experience (Gay & Howard, 2001), and the 
same can be said about the online learning environment (Wang, 2007). Taking these factors into account, Rice 
Doran et al (2011) suggested that SNA should be used to analyze the potential pattern of participation in the 
online learning environment. In conclusion, Rice Doran et al (2011) argued that when analyzing learners’ 
participation in an online discussion and learning environment, researchers should take various potential 
elements into account by using the closeness centrality of the number of posts rather than simply looking at the 
number of posts, which is consistent with the results of this study. 

Third, the difference between participation measured by closeness centrality and counting of meaningful posts 
was the observed power. According to Tables 2, 4, and 5, each MANOVA result and univariate test about 
feedback and SRL were statistically significant, but the observed power of the closeness centrality measurement 
was higher than that of the counting of meaningful posts measurement. This means that higher observed power 
implied lower type II error. In this way, SNA of posts is a credible and valid method compared with the general 
counting of meaningful posts, especially for the analysis of a complex and dimensional learning environment 
influenced by various factors. 
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Overall, these findings indicate that SNA measures can explain networked relationships more specifically in an 
online learning environment. Although counting meaningful posts can be used to analyze learners’ participation 
in an online learning environment, it is more important how much each actor plays a central and critical role in 
online discussions. Given the traits of the online learning environment, analysis of the central role in the 
environment can be verified by SNA measures, namely regarding the counting of meaningful posts, as learners’ 
participation does not include attributes of the online learning environment; however, SNA measures that 
consider various factors not overlooked can involve the traits of the online learning situation. 

Going forward given the diversity, complexity, and massive scale of big data, network visualization will 
facilitate data analysis. This can also be useful in analyzing data stored in the Learning Management System 
(LMS), which includes nonstructural and large-scale data, such as the learning activity status, posts left by 
learners, log-on times, study duration, and participation in team projects. For example, in Moodle-based LMS, 
the instructor may use display replies in unthreaded form for SNA. Therefore, in analyzing the LMS 
environment with accumulated big data, it is necessary to use visualized images of the network in addition to the 
basic statistics data. Through this, the network’s structure and characteristics in online discussions can be better 
understood. 
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